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AND TRIANGLE GROUP AVOIDANCE 
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Abstract. We study a family of infinite- type Coxeter groups denned by the avoid- 
ance of certain rank 3 parabolic subgroups. For this family, rationally smooth 
elements can be detected by looking at only a few coefficients of the Poincare 
polynomial. We also prove a factorization theorem for the Poincare polynomial 
of rationally smooth elements. As an application, we show that a large class of 
infinite-type Coxeter groups have only finitely many rationally smooth elements. 
Explicit enumerations and descriptions of these elements are given in special cases. 



1. Introduction 

Let W be a Coxeter group with finite generating reflection set S, and let £ and < 
denote the length function and Bruhat order on W, respectively. Let e G W denote 
the identity of W. By definition, W is the group generated by S satisfying relations 
(st) mst = e, where m st G {1, 2, 3, ... , oo} such that m st = 1 if and only if s — t. If 
m st = oo, then by convention the relation (st)°° = e is omitted. The Poincare series 

p«(?) = xy (x) 

x<w 

of an element w G W is a polynomial of degree £{w). An element w is said to be 
palindromic (or rationally smooth) if the coefficients of P w (q) are the same whether 
read from top degree to bottom degree, or in reversed In other words, if we write 
Pw{q) — J2 a iQ\ then w is palindromic when a, = ae( w )-i for all i. 

An important question in the combinatorics of Coxeter groups is to describe the 
set of palindromic elements of W. This question stems from its connection with 
the topology of Schubert varieties. A Coxeter group is crystallographic if m st G 
{2,3,4,6,oo} for all s 7^ t. If W is crystallographic, then it can be realized as 
the Weyl group of a Kac-Moody algebra. The Schubert subvarieties of the full flag 
variety corresponding to this algebra are indexed by the elements of W. Carrell 
and Peterson prove that the Schubert variety indexed by w is rationally smooth if 
and only if w is palindromic [9]. Furthermore, w is palindromic if and only if the 
Kazhdan-Lusztig polynomial indexed by (x,w) is equal to 1 for all x < w [HI |12] . 
If W is crystallographic, then it is sufficient that the Kazhdan-Lusztig polynomial 
indexed by (e, w) be equal to 1 0. For Schubert varieties of simply laced types A, D 
and E, the notion of smooth and rationally smooth are equivalent. For finite Weyl 
groups, the palindromic elements are well understood. In particular, they can be 



1 The term rationally smooth seems to be more common in the literature; we use the term 
palindromic to be inclusive of the non-crystallographic case. 
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characterized using permutation pattern-avoidance in classical types A, B, C, and D 
and using root system avoidance in all types [3J SI Ej, [13] . The characterization using 
permutation pattern avoidance has recently been extended to the affine type A case 
as well [2]. The generating series for the number of palindromic elements in A n , as 
n varies, is also known [3J [SJ [To] . 

While the theory of palindromic elements is well-developed for finite and affine 
Coxeter groups, the situation for general Coxeter groups is quite different. In par- 
ticular, it seems to be quite difficult to determine whether or not an element of a 
general Coxeter group is rationally smooth. In this paper, we introduce a family of 
Coxeter groups (mostly) outside the finite and affine cases, for which it is possible to 
determine if an element is rationally smooth by looking at just a few coefficients of 
the Poincare polynomial. The family in question is defined as the set of all Coxeter 
groups which do not contain certain triangle groups as standard parabolic subgroups. 
A triangle group is a Coxeter group with \S\ = 3. Triangle groups arise naturally 
in arithmetic geometry and the study of tessellations of triangles on Riemann sur- 
faces, see e.g. [TJ. We will denote a triangle group by the triple (m rs , m rt , m st ) where 
S = {r, s, t}. We say a Coxeter group W contains the triangle (a, b, c) if there exists a 
subset {r, s,t}CS such that (a, b, c) = (m rs ,m rt , m st ). If S contains no such subset, 
then we say W avoids the triangle (a, b, c). We are in interested in the groups which 
avoid the following special set of triangle groups: 

HQ := {(2, b, c) | b, c > 3 and b < oo} 

The set HQ (Hecke quotients) is the set of quotients of the Hecke triangle group 
(2,p, oo), p > 3, which is a generalization of the well-known modular group (2, 3, oo). 
Every finite Coxeter group of rank > 3 contains a triangle in HQ, and the same is 
true of affine Coxeter groups, with the exception of (3, 3, 3), which is the affine group 
Ai- However, there are many crystallographic Coxeter groups which do avoid HQ; 
for example, any Coxeter group with no commuting relations (i.e. m st > 3 for all 
s 7^ t) avoids HQ. Any Coxeter group defined by only by commuting and infinite 
relations also avoids HQ. 

To state our main theorem, we make the following definition: 

Definition 1.1. Let w be an element of a Coxeter group W , and write P w {<l) = 
Q>iq % for the Poincare polynomial of w. We say that w is ^-palindromic if ai = 
a<(iu)-i f or all < i < k . 

Note that if k = oo, then we recover the usual notion of palindromic elements, and 
that every element is 1-palindromic with ao = ai( w ) — 1- If W is crystallographic, 
then fc-palindromicity can be detected from the Kazhdan-Lusztig polynomial. Let 
T e , w = l+^i>o biQ 1 be the Kazhdan-Lusztig polynomial indexed by (e, w). A theorem 
of Bjorner and Ekedahl states that, for crystallographic groups, an element w e W 
is fc-palindromic if and only if 6j = for < i < k |7J (note that bo = always). 

We now state the main theorem: 

Theorem 1.2. Let W be a Coxeter group which avoids all triangle groups in HQ. 
Then every ^-palindromic w G W is palindromic. 
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Furthermore, ifW avoids all triangle groups (3,3, c) where 3 < c < oo ; then every 
2-palindromic w G W is palindromic. 

Given a Coxeter group, it is natural to ask whether there is a number k such that 
every ^-palindromic element is palindromic. This question appears to be open in 
general. Billey and Postnikov have conjectured that if W is a finite simply-laced 
Weyl group with n generators, then every (n + 1) -palindromic element of W is 
palindromic [4]. In type A n , it is known that every (n — l)-palindromic element is 
palindromic jl]. 

The proof of Theorem 11.21 is based on a factorization theorem for the Poincare 
polynomial of 2-palindromic elements in Coxeter groups which avoid HQ. In the 
classical groups of finite type A, B, C, and D, it is known that the Poincare polyno- 
mial of a rationally smooth element factors into a product of g-integers (see Equation 
©) 13 ED]- In fact, it is possible to see this factorization combinatorially, writing 
each palindromic element w as a reduced product W\ ■ • ■ W\su such that each g-integer 
factor of the Poincare polynomial equals the (relative) Poincare polynomial of the 
Wi's. We prove a similar result for 2-palindromic elements in Coxeter groups which 
avoid HQ. This result has a number of applications. For example, we show there 
are many infinitely large Coxeter groups with only a finite number of palindromic 
elements. We also give explicit descriptions of palindromic elements in special cases. 
In the case of uniform Coxeter groups W(m, n), defined by m st = m for all s ^ t and 
\S\ = n, we calculate the generating series for the number of palindromic elements 
weighted by length. Formulas for these generating series are stated in Propositions 
13.81 and 13.91 We also observe that the HQ-avoiding groups form the largest class of 
Coxeter groups for which our factorization theorem can hold. 

1.1. Organization. Section [2] contains some background material and elementary 
lemmas used to state the factorization theorem. Section [3] states the main fac- 
torization theorem and its consequences, including the proof of Theorem 11.21 and 
enumerative results. In Section HJ we consider triangle groups in the set HQ and 
prove the main results cannot hold for any Coxeter group containing these triangle 
groups. Section gives some elementary lemmas on the descent sets of Coxeter 
groups avoiding HQ. Finally, Section [H] proves the main factorization theorem. 



2. Background and terminology 

Let W be a Coxeter group with simple generator set S. For basic facts on Coxeter 
groups, we refer the reader to [6]. Let £{w) denote the length of w G W. We say 
w = uv G W is a reduced factorization if £{w) = £(u) + £(v). A special type of 
reduced factorization can be constructed from any subset JCS. Let Wj denote the 
standard parabolic subgroup of W generated by J. Let W J denote the set of minimal 
length coset representatives of W^W 7 ! Every element w G W can be written uniquely 
as w = uv where u G Wj, v G W J and £{w) = £{u) + £{v). We call this reduced 
factorization of w the parabolic decomposition with respect to J. 
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Let < denote the Bruhat order on W. If u < v G W, then the interval [u, v] denotes 
the set of elements x G W such that u < x < v. For any w G W we can define the 
Poincare polynomial 

PM ■= E ^ x) - 

xd\e,w\ 

The Poincare polynomial relative to J C S of an element w G W is defined to be 

z€[e,to]rW J 

If w G W 7,7 , then P^(q) is a polynomial of degree £{w). If J = 0, then P^(q) = 
P w (q). Recall that for any J, the poset [e, w] fl WO has a unique maximal element. 
The following proposition is due to Billey and Postnikov in jH Theorem 6.4]. 

Proposition 2.1 Q4J). Let JCS and let w = uv be a parabolic decomposition with 
respect to J. Then u is the unique maximal element of [e, w] Pi Wj if and only if 

PM = Pu(q)-P v J (q). 

While the proof of Proposition 12. II given in jl] is stated only for finite Weyl groups, 
it easily extends to all Coxeter groups. A parabolic decomposition w = uv is called 
a BP-decomposition of w if u is the unique maximal element of [e, w] (1 Wj. 

For any w G W, define the sets 



s 




:={u 


< w 


i(u) 


= i} 


D 


» 


:={u 


< w 


£(u) 


= £(w) - 


D R 


» 


:={s 


G S | 


e(ws) 


< £(w)} 


D L 




:={s 


G S | 




< £(w)} 



The sets S(w) and D(w) are known as the support and divisor sets of w. The sets 
Dr{w) and Dl(w) are called the right and left descent sets of w respectively and 
are contained in S(w). We use these sets to give an equivalent characterization of a 
BP-decomposition. 

Lemma 2.2. A parabolic decomposition w = uv is a BP-decomposition if and only 
if S(v)n JC D R (u). 

Proof. If w — uv is a BP-decomposition, then u is the unique longest element of 
[e,w] fl Wj. If there exists x G S(v) fl J and x ^ Dr(u), then £(ux) = £{u) + 1 and 
ux G [e, w] fl PVj which is a contradiction. 

Conversely, assume that S(v ) fl J C Dr(u) and let m denote the maximal element 
in [e, u>] fl IVj. Since u is unique, we have that u <u. We now show that u <u. Let 

u = u'v' 

be a reduced factorization which maximizes £(u') under the conditions that u' < 
u and v' < v. Suppose that v' ^ e. Then there exists y G D L {y')\D R {u'). By 
assumption, we have that y G Dr(u). Taking a reduced decomposition for u with 
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y appearing at the end, we see that u' < uy, and hence u' can be extended, a 
contradiction. 

□ 

We remark that one direction of Lemma LT21 is proved in (T5J Lemma 10]. Another 
property of BP-decompositions is the following lemma. 

Lemma 2.3. Let J\ C J 2 C S and let V1V2V3 be a reduced factorization such that V\V 2 
and (viV2)(v3) are BP-decompositions with respect to J\ and J 2 respectively. Then 
^1(^2^3) is a BP-decomposition with respect to J\. 

Proof. By definition, we have V1V2 is maximal in [e, V1V2V3] fl Wj 2 . In particular, if u 
denotes the maximal element in [e,viv 2 v 3 ] D Wj 1} then u < v^ 2 since Wj 1 C Wj 2 . 
But now u is maximal in [e, V\V 2 \ D which implies that u = v\. □ 

Clearly, if P w (q) = Yli a i ( ti then |S(iu)| = a% and | D(w)| = at( w )-i- We now 
consider a special class of parabolic decompositions. 

Definition 2.4. We say that w = uv, a parabolic decomposition with respect to J, 
is a Grassmannian factorization if J = S(u) and \ S(w) \ = \ S(u) \ + 1 

It is easy to see that every element w G W of length > 2 has a Grassmannian fac- 
torization. The term "Grassmannian" comes from the fact that v is a Grassmannian 
element of W which, by definition, has \Dl(v) \ = 1. Note that a Grassmannian fac- 
torization is not necessarily a BP-decomposition. Although elementary, this concept 
is quite useful. For example, we can use it to prove: 

Lemma 2.5. | D(w)\ > \ S(w)\. 

Proof. We proceed by induction on £(w). The proposition is true if £(w) = 1, so 
suppose £(w) > 2. Let w = uv be a Grassmannian factorization with respect to J. 
By induction, | D(u)\ > \ S(u)\. 

If u' G D(w), then u'v G D(u>), since v G W J . Now v is not the identity, so we 
can write v' — v s G W J with s G S and £(v r ) = £{v) — 1. Consequently uv' G D(w). 
Moreover, uv' 7^ u'v for any u' G D(u) since they are both parabolic decompositions 
with respect to J and u ^ u'. Hence 

(1) I D(w)\ > I D(u)\ + 1 > I S(u)\ + 1 = | S(w)\. 

This completes the proof. □ 

We remark that, for crystallographic Coxeter groups, Bjorner and Ekedahl prove 
a much stronger version of Lemma 12.51 concerning all the coefficients of P w {q) [7J 
Theorem A]. 

We can continue to decompose any Grassmannian factorization w = uv by taking 
a Grassmannian factorization of u. We say that 

W = V X V 2 - ■ -V|S(w)| 

is a complete Grassmannian factorization of w if for every i < | S(u>)|, we have that 
(vi ■ ■ -Vi)(vi + i) is a Grassmannian factorization. Observe that if each (vi ■ ■ ■ Vi)(vi + i) 
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is also a BP-decomposition, then by Lemma |2~3| we have {y\ • ■ ■ v j)(t> j + i • • • Vk) is BP 
decomposition for any % < k < | S(u>)|. 

By definition, w is 2-palindromic if and only if | D(u>)| = | S(w)|. The following 
lemma gives an inductive characterization of the 2-palindromic property. 

Lemma 2.6. Suppose that w = uv is a Grassmannian factorization. Then w is 
2-palindromic if and only if u is 2-palindromic and \u ■ D(t>) fl D(u>)| = 1. 

Proof. Equality holds in Equation ([1]) if and only if | D(u)\ = \ S(u)\ and u ■ D(t>) fl 
D(w) = {uvs} where s G Dr{v). □ 

3. The factorization theorem 
The main technical theorem of this paper is the following: 

Theorem 3.1. Suppose that W avoids all triangle groups in HQ. Let w G W be 
2-palindromic and fix a Grassmannian factorization w = uv with respect to J C S. 
Then w = uv is a BP-decomposition with respect to J such that | S(t>)| < 3. 
Moreover, if \ S(v )| = 3 and S(v) = {r, s, t}, then one of the following is true: 

(1) v = trv' with v' = §tst^ . . . y where S(v) generates the triangle group (3, m rs , m st ) 

m s t — l 

with m rt = 3 and 3 < m st < oo, 3 < m rs < oo. 

(2) v = rstrv' with v' = stst . . . where S(v) generates the triangle group (3, 3, m st ) 

m st — 1 

with 3 < m st < oo. 

(3) v = strstr ■ ■ ■ is a spiral wora^ of even length where S(v) generates the tri- 
angle group (3,3,3). 

Theorem 13.11 says that if W avoids triangle groups in HQ, then the Poincare 
polynomial P w {q) of a 2-palindromic element w G W factors along any Grassmannian 
factorization of w = uv . Moreover, the possibilities for the factor P^{q) is limited 
by the fact that | S(t> )| < 3. Note that parts (1) and (3) of the theorem overlap when 
m rs = m st = 3. The proof of this theorem is the focus of Section [6j The remainder 
of this section is devoted to consequences of Theorem 13.11 

Fix a 2-palindromic element w G W and a Grassmannian factorization w = uv 
with respect to J C S. Theorem 13.11 can be used, together with Lemma 12.6} to 
completely determine the polynomial P w (q). By Theorem 13.11 and Proposition 12. 1^ 
we have that 

P w (q)=P u (q)-P v J (q), 
so it suffices to characterize all possible polynomials P/(g). For any integer k > 1 
define the g-integer 

(2) [k] q :=l + q--- + q k -\ 

If | S(v)\ < 2, then any v' < v where v' G W J is given by a prefix of the unique 
reduced word of v. This implies 

(3) P v J (q) = [£(v) + 1],. 



'A spiral word is word which cycles through a set of generators in a fixed order. 
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If | S(v)\ = 3, it suffices to compute Py(q) in all the cases of Theorem 13.11 We 
have the following lemma. 

Lemma 3.2. Suppose we have w = uv as in Theorem \3 . 1\ with | S(t>)| = 3. Then the 
following are true: 

(1) If v satisfies the conditions in Theorem \3. 1\ part (1), then 

Pv J (q) = m + l} q + q 2 {£(v)-3} q . 

(2) If v satisfies the conditions in Theorem \3. 1\ part (2), then 

P v J (q) = [i(v) + 1], + q 2 [£(v) - 3] q + q 4 [£(v) - 6} q . 



(3) If v satisfies the conditions in Theorem \3. 1\ part (3) with k 



£(v) 



, then 



J?(g) = £g*[*(t;)-4i + l],. 

i=0 

Proof. Part (3) is proved in [T4J Proposition 2.4] where certain Poincare polynomials 
of Schubert varieties in the affine Grassmannian of type A are calculated. Parts (1) 
and (2) can be deduced from elementary counting arguments of the sets 

{«' G W J f] [e,v] | £(v') = i}. 

In particular, for part (1), there are two g-integer contributions from reduced sub- 
words of the form 

tr stst^. . . f and fstst . . > . 

k k 

For part (2) there are three g-integer contributions from reduced subwords of the 
form 

r ^sts . . . f and r stst . . . and rstr $tst . . . J . 

k k k 

□ 

The polynomials in parts (1) and (3) of the lemma are palindromic, while the 
polynomial is part (2) is 3-palindromic but not 4-palindromic. We now prove the 
theorem stated in the introduction. 

Proof of Theorem 11.21 Suppose that W avoids all triangles in HQ . Let w = 
v\v 2 ■ ■ ■ v\s( w )\ £ W be a complete Grassmannian factorization. Then by Theorem 
13.11 and Proposition 12.11 we have that 

SMI 

pm = n p i (?)■ 

i=i 

where Jj := S(fi)U- • -US(fj_i) and J\ := 0. Moreover, the factors P^{q) are given by 
either Equation ([3]) or by parts (l)-(3) of Lemma \3. 21 Now P w {q) is 4-palindromic if 
the polynomial in Lemma [3.21 part (2) does not appear as one of the factors P^(q). 
Since all other possible choices for P^ l (q) are palindromic, we have that P w {q) is 
4-palindromic if and only if it is palindromic. This proves part (1) of Theorem 11.21 
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If W also avoids the triangles of the form (3, 3,c), then Lemma 13.21 part (2) is 
never an option for P£(q). Hence every 2-palindromic w G W is palindromic. This 
completes the proof. □ 

3.1. Examples. Consider the Coxeter group W with S = {si, s 2 , s 3 , s 4 } defined by 
the Dynkin diagram in Figure [TJ 




Figure 1. 

Unlabeled edges are assumed to have label m st = 3 and if there is no edge between 
s and t, then m st = 2. Clearly, W avoids all triangle groups in HQ and hence we can 
apply Theorem 13.11 to compute Poincare polynomials. 

Example 3.3. Let w = S1S2S1S3S2S1S3S2S1S4. Then w is 2-palindromic with \ S(w)\ = 
I D(iu)| = 4. The following is a complete Grassmannian factorization: 




V\ V2 V3 



The corresponding Poincare polynomial factorization is 
P w {q) = [2} q \%{\7} q + q 2 [^ q )[2} q 

= (1 + q)(l + q + q 2 )(l + q + 2q 2 + 2q 3 + 2q A + q 5 + g 6 )(l + q), 
so P w (q) is palindromic. 

Example 3.4. Letw = s 2 S4S2S4SiS2S4SiS 2 S4S2. Thenw is 2-palindromic with \ S(w)\ = 
I D(w)| = 3. A complete Grassmannian factorization of w is 

W = (s 2 ) (S4S2S4) (S1S2S4S1S2S4S2) • 

VI V2 V3 

The corresponding Poincare polynomial factorization is 
P w (q) = [2} q [A} q ([8} q + q 2 [A} q + q A [l} q ) 

= (1 + q)(l + q + q 2 + q 3 )(l + q + 2q 2 + 2q 3 + 3q A + 2q 5 + q 6 + q 7 ) 



Note that {si, 32,34} generates the triangle group (3,3,4). Since t> 3 = S1S2S4S1S2S4S2, 
we have that w is 3-palindromic but not ^-palindromic. 

An example of a HQ-avoiding Coxeter group W with commuting relations is given 
by the Dynkin diagram in Figure H] below where p > 3. 

Observe that W also avoids all triangle groups of the form (3,3,c). Hence every 
2-palindromic element is palindromic by Theorem 11.21 Moreover, every palindromic 
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Figure 2. 

polynomial factors into a product of g-integers. We also remark that W is indecom- 
posable with respect to products and free products of Coxeter groups. 

3.2. Enumeration and description of palindromic elements. Theorem 13.11 
gives a description of the set of palindromic (resp. 2-palindromic) elements of any 
HQ-avoiding Coxeter group. Specifically, the palindromic (resp. 2-palindromic) ele- 
ments are those with a certain Grassmannian factorization. In this section we provide 
some applications of this idea. We start by proving a corollary of Theorem 13.11 on 
the finiteness of the number of palindromic elements for all HQ-avoiding Coxeter 
groups. 

Corollary 3.5. Let (W, S) be a Coxeter group that avoids all triangle groups in HQ . 
Then W has a finite number of palindromic elements if and only if m st < oo for all 
s,t G S and W avoids the triangle group (3,3,3). 

Proof. Theorem 13. II part (3) implies that the triangle group (3,3,3) contains an infi- 
nite number of palindromic elements. Also, if m rs = oo, then Wi r a \ is infinite and 
every element is palindromic. 

Let mo denote the largest value of m st for s,t 6 S. Suppose that W avoids (3, 3, 3) 
and mo < oo. Let w G W be palindromic with complete Grassmannian factorization 
w — V\ ■ • ■ui\s{w)\- By Theorem 13. 1[ we have that each factor has length at most 
m + 3, so 

£{w) < | S{w)\{m + 3) < IS] (mo + 3) 
and hence the number of palindromic elements in W is finite. □ 

Corollary 13.51 also holds if palindromic is replaced by 2-palindromic. 

Note that the Grassmannian factorization of an element provided by Theorem 13.11 
is not necessarily unique. When m s t > 3 for all s ^ t, we give a modified factorization 
which does not have this problem. To state the modified factorization we need the 
following definition. 

Definition 3.6. We say a reduced factorization w = u\Ui ■ ■ - Ud is separable i/S(uj)n 
S(uj) = for all i ^ j. If no such non-trivial factorization exists, then we say that 
w is inseparable. 

Given any complete Grassmannian factorization of a palindromic element w = 
v i • • • ^|S(u))|, there is a simple method for constructing a separable factorization. Let 
. . . ,id) denote the subsequence of integers for which = 1. Then w — U\ ■ ■ - Ud 
is a separable factorization where 



Uj := Vi.v ij+1 ■ ■ -v ij+1 -i 
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and id+i '■= | S(w)| + 1. We remark that i(v\) = 1 and hence the sequence (zi, ■ ■ ■ ,id) 
is nonempty. Furthermore, each factor Uj is inseparable. For example, let W be 
defined by the Dynkin diagram in Figured] and w = S4S2S4S2S3S1S3. Then w = u\Ui 
given by 



is a separable factorization. The following corollary follows from Theorem 13.11 

Corollary 3.7. Let W be a Coxeter group with m st > 3 for all s 7^ t, and let 
w G W be palindromic. Then w has a unique separable factorization w = u\- ■ -Ud 
where each it, is inseparable and palindromic. Moreover, any complete Grassmannian 
factorization = V\ • • •U|s(u < )| is unique up to choice of V\. 

Proof. Any element w has a separable factorization w = u\ - ■ - Ud where each Ui 
is inseparable. Since S{uj) is distinct and W has no commuting braid relations, 
the factorization is unique. If w is palindromic, then every Ui is palindromic since 
(ui ■ ■ ■ Ui){ui + i) is a BP-decomposition with respect to J = S\ S(ui). 

Let Ui = vi ■ ■ ■ v\s( Ui )\ be a complete Grassmannian factorization, and let Sj be 
the unique element of D^Vj). Note that v\ = s\. As mentioned above, since Ui is 
inseparable, we must have | S{vj)\ > 2 for j = 2, . . . , | S(ttj)|. Indeed, if S(vj) = {sj} 
then Sj is the unique right descent of V\---Vj, since Sj $ S(vi ■ ■ ■ v But by 
Lemma [2~5| [v\ ■ • -Vj)(vj + i ■ ■ •fis(t ti )i) is a BP decomposition, so 

S(v j+1 . . .«|s(«i)|) n S(«i • -Vj) C {sj}. 
Thus (vi ■ ■ -Vj^ijivj ■ ■ ■v\s( Ui )\) is a separable factorization, which is a contradiction. 
We now show that Sj is the unique left descent of Vj • • ~v\s(ui)\i 

for j > 2. Indeed, 

looking ahead to Lemma |5.3[ and using the fact that |S(uj)| > 2, we see that 
Di(vj ■ ■ ■ v 1 s(ui)\) is a subset of S(vj) \ S(v\ • • • Uj-i) = {sj}. Hence the sequence 
(s 2 , • • • , S|s( Ui )|) is uniquely determined given the choice of v% = s%, and the v/s are 
uniquely determined from the corresponding parabolic decomposition. 



Note that there are at most two complete Grassmannian factorizations of each 
in Corollary 13.71 For example, taking u\ in Equation 01]), we have 



as the only two complete Grassmannian factorizations. 

Corollary 13.71 implies that to count the number of palindromic elements of W, 
it is sufficient to enumerate elements of W which are inseparable and palindromic. 
When m st is constant we compute an exponential generating series for the number 
of palindromic elements. Specifically, let W(m,n) denote the uniform Coxeter group 
such that \S\ = n and m st = m for all s 7^ t. Uniform Coxeter groups satisfy the 



(4) 




W = S 4 S2S4S2 S3 S1S3 




□ 



U\ = S4 S2S4S2 = S 2 S4S2S4 
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property that every 2-palindromic element w is palindromic by Theorem 13.11 Define 
the generating series 

®m(q,t) := Y] Pn,k j- 

z — ' n! 

n,fc>0 

where P n ^ denotes the number of palindromic w G W(m,n) of length k. In the 
case that m = 2, we have W(2,n) ~ (Z/2Z) n with every element palindromic, so 
P n ,k = (k) ■ Hence the generating series 

$ 2 (q,t) = exp(qt + t). 

For m > 3, define 

q k t n 



■■= ^ In,k 



n.k>l 



rv. 



where denotes the number of palindromic w G W(m,n) of length k that are 
inseparable with | S(w) \ = n. Note that $ m and m are exponential in t and ordinary 
in q. Corollary 13.71 implies 

Proposition 3.8. For any 3 < m < oo, the series 

exp(t) 



1 - <j>m(q,t)' 

The following proposition completes the calculation. 

Proposition 3.9. The exponential generating series for the number of inseparable 
palindromic elements in W(m, n) is 



(5) 



<frm(q,t) 



(2q - 2q 3 )t - (3g 3 + q 5 )t 2 
2 - 2q 2 - 4q 2 t 

2qt - 3q m t 2 - q m+2 [m - 3]^ 3 
2 - 2q 2 t([m - 2} q + q 



qt — q 2 t 



m—3\ 



for m = 3 
for 4 < m < oo 
for m = oo. 



1 — q — q 2 t 

Proof. By Theorem 13.11 \Dr(w)\ < 2 for any palindromic w G W(m,n). Hence we 
can partition the set of inseparable palindromic elements into those with \Dr(w)\ = 
1, 2 respectively. For notation, let A n ^ be the number of inseparable palindromic 
w G W(m, n) of length k with | S(w)\ = n and Dr{w) = 1. Let B n ^ be the number 
of those same elements with Dr(w) = 2. We have = A n ^ + B n ^. Consider the 
polynomials 

A n (q) ■■= -^tV] A n , k q k and B n (q) := — V £ n , fc 



fe>i 



fc>i 



If n = 1, then 



^i(g) = g and £i(<?) = 0. 
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If 3 < m < oo, then for n = 2, the inseparable elements have the form S1S2S1 • • • 
or S2S1S2 • • • where the length is at least 3. There is also a unique longest element 
Wq := S1S2 • • • with \Dji(wo) \ = 2. This gives 



M{q) = q 3 [m - 3] g and 



o'" 



For the remainder of the proof, let w = v% ■ • • v\ s( w )\ £ W(m, n) be a complete Grass- 
mannian factorization. We first consider the case when m = 3. If w is palindromic 
and inseparable, then by Theorem 13. 1[ each V{ is a spiral word as in Theorem 13 . 1 1 part 
3. In particular, for each even length, there is a unique Vj of up to 5*3 permutation 
symmetry on the generators {r,s,t}. Moreover, if |S(w)| > 3, then |.D£,(iy)| = 2. 
Thus for all n > 3, we have A n (q) = and 

■><,- \ „ ^ <[' ( 2q 2 x "- 1 



B n (q) 



Hence 



Bn-M = \ 



1 . , q 3 t 2 q 5 / 2q 2 



n>3 



n— 3 



This proves the first equation in flSJ). 

Now suppose 4 < m < 00. In this case, if w is palindromic and inseparable, then 
Theorem 13.11 implies | S(vi)\ < 2. Hence each factor Vi has a reduce expression stst ■ ■ ■ 
where t G Dl{v\ ■ ■ ■ In particular, when constructing w — V\ ■ ■ • v\ s(w)\, there are 
exactly twice as many choices for Vi if Dl{v\ ■ ■ ■ i>j_i) = 2 than if Dl(v\ ■ ■ ■ fj-i) = 1. 
This yields that for n > 3, the polynomials A n (q) and B n (q) satisfy the first order 
recurrence 



A n {q) = q 2 [m - 3} q (A n ^{q) + 2£? n _ 1 (g)) 
B n (q) = q m - 1 (A n „ 1 (q) + 2B n _ 1 {q)). 



This implies that 



A n {q) 




W-3] g 2q 2 [m-3] q ~ 


n-2 


Mi)' 


B n (q) 




q m-i 2q m ~ 1 




B 2 (q) 



m—l\n—3 



q 5 [m-2] q (q 2 [m-3] q + 2q m - 1 ) 



\m 



-3], 

jm— 3 



Thus 



q 7 [ m -2} q ([m-2] q + q 



m—2\n—3 



[m - 3], 



-,m— 3 



m (g, *) = qt + ( g 3 [m - 3], + H — ) t 2 + q 7 [m - 2] 2 ]T ([m - 2] q + q ^) n ~ 3 t n 



n>3 



which proves the second equation in © 
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Finally, we compute the exponential generating series for the uniform Coxeter 
group W(oo,n) by taking the limit of m in the second equation of (j3j) as m — > oo. 
This is equivalent to taking 

q m ->■ and [m\ q -> ^ — 

which yields the third equation in (J5]). □ 

The following equations are the first few terms in the Taylor expansion of $ m ((?, t) 
for m = 3,4, oo. These calculations were computed using the combinat package for 
Mupad. 

t 2 

<S> 3 (q,t) = l + (l + q)t+(l + 2q + 2q 2 + q 3 )- 

t 3 

+ (1 + 3q + 6q 2 + 9g 3 + 6g 4 + 6q 5 + 6q 7 + 0(q 9 )) - 

o 

t 4 

+ (1 + Aq + 12g 2 + 30g 3 + 48g 4 + 60g 5 + 54g 6 + 0(g 7 )) — + 0(f) 
$ 4 (g,t) = l + (l + g)t+(l + 2g + 2g 2 + 2g 3 + g 4 )| 

+3 

+ (1 + 3q + 6g 2 + 12g 3 + 15g 4 + 12g 5 + 12g 6 + 6q 7 ) - 

6 

+ (1 + 4g + 12g 2 + 36g 3 + 78q 4 + 120g 5 

t A 

+ 156g 6 + 168g 7 + 150g 8 + 120g 9 + 48g 10 ) — + 0{f) 

t 2 

$oo(g,t) = l + (l + g)t + (l + 2g + 2g 2 + 2g 3 + 2g 4 + 2g 5 + 0(g 6 ))- 

t 3 

+ (1 + 3q + 6q 2 + 12q 3 + 18g 4 + 24g 5 + 0(g 6 )) - 

o 

t A 

+ (1 + Aq + 12g 2 + 36g 3 + 84g 4 + 156g 5 + 0(g 6 )) — + 0(t 5 ) 

By evaluating <& m (q,t) at g = 1, we can recover the total number of palindromic 
elements in W(m,n). By Corollary 13. 5[ this value is finite only when 4 < m < oo. 
We list these values for 4 < m < 8 and 1 < n < 7 in Figure [3j 

4. Properties of triangle groups in HQ 

We discuss a few properties of triangle groups in HQ . The first property is that 
there are /c-palindromic Poincare polynomials which are not palindromic for large k: 

Proposition 4.1. Let W be the triangle group (2, b, c) with S = {r, s,t} such that 

( rs ) 2 = (rt) b = (st) c = e 
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m\n 


1 


2 


3 


4 


5 


6 


7 


4 


2 


8 


67 


893 


15596 


330082 


8165963 


5 


2 


10 


115 


2057 


47356 


1314292 


42584795 


6 


2 


12 


175 


3893 


110436 


3768982 


150113447 


7 


2 


14 


247 


6545 


219956 


8884312 


418725119 


8 


2 


16 


331 


10157 


393916 


18351562 


997538291 



Figure 3. Number of palindromic elements in W(m,n) 

where b, c > 3 and c is finite. Then there exist elements w G W which are (c — 2)- 
palindromic but not palindromic. 

Proof. Consider w = uv where 

(6) u~ l = stst . . . and v = rtstst . . . 

with £{u) < c and £(v) < c. Calculation of the polynomial P w (q) reduces to deter- 
mining the cardinality of the sets 

M k ■= {w'<w\ £{w') = k}. 

First we partition 

M k = (M k n w {Stt} ) u (M k n w\w {Stt} ). 

If w' G W{ s j}, then w' has the form sts ■ ■ ■ or tst ■ ■ ■ . Hence 

{2 if k < min{c,£(w) - 2} 
1 if k = c or £{w) - 1 
if k > c. 

If w' G W\W{ St t}, then it is uniquely determined by its parabolic decomposition 
w' = u'v' where v! < u, v' < v and v' is nontrivial in W^ s,t ^ . Hence 



W\W {Stt} ~ {e,u} x ({r,v}nW {s > t} ). 



This gives that 

\M k n w\w {Sit} 



(2k -l if k <min{£{u),£{v)} 

2£(v) if £{v) <k< £{u) 

2£{u) if £{u) <k< £{v) 

2£(w)-2k + l if k > m&x{£(u) , £(v)} 

In the case that £(u) = £(v) = c — 1, we have that 



PM = [*W + 1]« + ? C+1 + I]2g fe [£H -2k + 1], 



k=i 



In particular, if we write P w (q) = a iQ\ then we have that 

a-i = ae(w)-i = 2i + 1 
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for % < c — 3 and 

a c _2 = 2c — 1, a c = 2c — 2. 

Hence P w {q) is (c — 2)-palindromic but not palindromic. For example, if we take 
c = 4 and w = uv = (sts)(rts), then 

[e, w] fl W^s.t} = {e, s, t, st, ts, tst, sts, stst} 

and 

[e, iu] n W\W {Sjt} = [e, u] • ([r, u] n = {e, s, t, st, ts, sts} ■ {r, rt, rts}. 

In this case, the Poincare polynomial P w (q) = 1 + 3q + 5q 2 + 7q 3 + 6g 4 + 3g 5 + q 6 is 
2-palindromic but not 3-palindromic. □ 

It is tempting to conjecture that, for the triangle groups (2, b, c) as in Proposition 
14.11 a h ( c ~ l)-palindromic elements are palindromic. However for triangle group 
(2, 3, 5) (Coxeter type with c = 5) there is a unique length 14 element which is 
4-palindromic but not palindromic given by w = tsrtsrtsrtsrtr. 

Theorem 13.11 states that any Grassmannian factorization of a 2-palindromic ele- 
ment w G W is also a BP-decomposition if W avoids triangles in HQ. This statement 
is not true for Coxeter groups which contain triangles in HQ. 

Proposition 4.2. Let W be a Coxeter group. Then W avoids all triangle groups in 
HQ if and only if every Grassmannian factorization w = uv where w is palindromic 
is a BP-decomposition. 

Proof. By Theorem I3.1[ it suffices to show that for triangle groups (2, b, c) as in 
Proposition 14.11 there are Grassmannian factorizations w — uv of palindromic w 
which are not BP-decompositions. Consider w = uv as in Equation fl6]) with £(u) = 2 
and £(v) = c = m st . It is easy to check that w is palindromic and that w = uv is a 
Grassmannian factorization with respect to J = {s,t} but not a BP-decomposition. 

□ 

5. Descent sets of triangle avoiding groups 

In this section, we prove several basic properties of Coxeter groups which avoid 
triangle groups in HQ . We begin with a lemma that holds for all Coxeter groups: 

Lemma 5.1. Let W be a Coxeter group andu G W . If s £ Dl(u), then Dl(su)\{s} 
consists of the elements t G Dl(u) such that u has a reduced factorization starting 
with a braid tsts ■ ■ ■ of length m st — 1. (If m st = 2 then this braid consists of only 
one element.) In other words, 

D L (su) = {s} U{te D L (u) : u = u qUi ,u G W {s>t} ,Ux G W {s ' t} , £(u ) = m st - 1}. 

Proof. Let J = Dl(su). Then by [6], Wj is a finite Coxeter group and su has 
a reduced factorization beginning with the maximal element wq of Wj. If t is an 
element of J \ {s}, then m st < oo and wo has a reduced decomposition starting with 
the longest element of W{ S)t }- □ 

We now consider Coxeter groups which avoid triangle groups in HQ . 
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Lemma 5.2. If W is a Coxeter group which avoids all triangle groups in HQ, then 
the only finite parabolic subgroups of W are products of rank 2 Coxeter groups. 

In other words, if J C S is such that Wj is finite, then J can be written as a 
disjoint union 

J = |Jj, 

i 

where | Jj| < 2 for all i, and m st = 2 if s G Jj, t G Jj, i ^ j . 

Proof. Using the classification of finite Coxeter groups, we see that every finite irre- 
ducible Coxeter group of rank > 3 contains a triangle group in HQ. □ 

If J = D L (w), then Wj is a finite Coxeter group. In particular, Lemma I5T21 applies 
to the parabolic subgroups generated by descent sets of HQ-avoiding Coxeter groups. 
The following lemma is the main result of this section. 

Lemma 5.3. Let (W, S) be a Coxeter group which avoids triangle groups in HQ. Let 
r, s G S such that 3 < m rs < oo, and suppose u is an element ofW such that (rs)u 
is a reduced factorization. Then 

D L (rsu) \ {r, s} = {t G D L (u) : m rt = m st = 2}. 

Proof. The proposition is obviously true if u — e. We proceed by induction on the 
length of u. Let J = Dl(su), and write J = \_\ Ji as in Lemma l5\2l We can further 
assume that if Jj = {x, y}, then m xy > 3, and that s G Jo- 

Now if t G Di{rsu) \ {r, s}, then by Lemma [5.11 we must have m rt < oo and rsu 
must have a reduced decomposition starting the longest element in W{ r>t }. If t Jo 
then m st = 2. Since W avoids triangle groups in HQ, we have that m rt = 2 as well. 

This leaves the possibility that t G Jo, in which case m st > 3. Once again, since 
W avoids triangle groups in HQ, we conclude that m rt > 3. Thus rsu has a reduced 
factorization rsu = (rtr)u', where £(u') = £(u) — 1. Now tru' = su, so s G D^tru'). 
But by induction, this implies that m ts = m rs = 2, which is a contradiction. Hence 
t i J . □ 

6. Proof of Theorem 13.11 

We now prove Theorem 13.11 The following assumptions are fixed for the remainder 
of the section. Let W be a Coxeter group that avoids all triangle groups in HQ . Let 
w G W be 2-palindromic with a Grassmannian factorization w = uv with respect to 
J = S(u). By Lemma [2.61 we have that 

(7) \u ■ D(v) n D(w)\ = I. 
This implies 

(8) \W J f] D(v)\ = 1, 

and in particular, \D R (v)\ = 1. Let z G D R {y) denote this unique simple reflection. 
The element vz is the unique element in W J R D(v ). 

We divide the proof into three steps. The first step is to prove that S(v ) has 
at most three elements. Second, we prove the characterization of v when S(v) has 



COXETER GROUPS AND TRIANGLE AVOIDANCE 17 

exactly three elements. For the last step, we show that w = uv is BP-decomposition. 
We begin with the following technical lemma. 

Lemma 6.1. Let sx,...,Sk be the longest sequence of distinct simple reflections such 
that v has a reduced decomposition 

V = Si ■ --SkV 1 , 

and for all j < k, m SjSj+1 > 3. For any 1 < j < k, define the set Ij := {si, . . . , Sj}. 
Then: 

(1) Sj ■ ■ ■ s k v' G W 1 ^ 1 for all j < k, and 

(2) S(u') C { Sl ,...,s k }. 

Proof. Clearly the lemma is true if £(v) = 1 and hence we assume that £(v) > 2. 
Observe that k > 2, otherwise v ^ W J . For any j < k, let 

be a parabolic decomposition with respect to Wj j . It is easy to see that v\ = s\ and 
hence v[ — s 2 • ■ ■ s^v' G W Ix . 

Now let j > 2 and suppose that £(vj) > j. Then there exists s G Dr{vj) such that 

S(vjs) = S(vj). 

By Lemma I5.3[ we have that the left descent sets 

D L (v jS v^) = D L {v jS ) U {t G D L {v' 3 ) | m tSi = 2 for i < j} 

and 

D L (v) = D L ( Vj v^) = D L ( Vj ) U {t G D L (v^) | m tSi = 2 for i < j}. 

Since S(vjs) = S(vj), the descent sets above are equal. Hence Vjsv'j G W J fl D(t>). If 
j < k, then v'j ^ e and consequently Vjsv', ^ vz, contradicting Equation Thus 
£(vj) = j which implies that vj — si • • • Sj and v'j = Sj + i ■ ■ ■ s k v' G W Ij . This proves 
part (1) of the lemma. 

For part (2), suppose that S(f') ^ {s\, . . . , Sk}. Then | S(v)\ > k and v' k ^ e. We 
get that Vk = s\ - ■ ■ Sk and 

D L (v' k ) fl {s h ■ ■ ■ , s k } = 

since v' k G W Ik . By the maximality of k, we have that m Skjr = 2 for all r G DL(v' k ). 
We claim that 

L> L (si • ■ ■ s k -iv') = D L (v). 
Indeed, if k > 3, this follows from Lemma [5.31 Otherwise, if = 2, then 

since avoids all triangle groups in HQ . This proves the claim when k — 2. In 
either case we have that 

si---s fc _it/ G lf J nD(i;) 
which contradicts Equation Therefore S(v') C {s x , . . . , s^}. □ 

The following proposition completes the first step of in the proof of Theorem 13. 11 
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Proposition 6.2. We have that \ S(v)\ < 3. Furthermore, if \ S(v)\ = 3, then S(v) 
generates a triangle group (a, b, c) with a,b,c> 3. 

Proof. Suppose | S(t>)| > 4 and let v — si ■ ■ • Skv' as in Lemma \6. II We first show by 
induction on j that 

(1) D L (sj ■ ■ • s k v') = {sj} 

(2) m SiS . = 2forze{l,...,j-2}. 

Indeed, part (1) is trivial for j = k. Suppose part (1) is true for some j < k. 
Now by Lemma 16.11 s± ■ ■ ■ Sj- 2 Sj ■ ■ ■ Skv' is reduced, and therefore is not an element 
of W J . So by Lemma 15.31 we have that Sj G D L (s\ ■ ■ ■ Sj^ 2 Sj ■ ■ ■ s^v'). Moreover, if 
j > 4, then m SiSj = 2 for all 1 < i < j — 2. If j = 3, then S1S3 ■ ■ ■ SkV ' has a 
reduced expression beginning with a braid S1S3S1 ■ ■ • of length m SlS3 < 00. Since 
s 1 ^ -D_l(s 4 • ■ ■ Sfcf'), we conclude that m SlS3 = 2. Hence part (2) holds for j. 

Now suppose part (2) holds for all j > j . Since \D L (v)\ = 1, we have that 
Sj £ D L (sj Q ■ ■ ■ s^v') for any j > j . Thus part (1) holds for j . Hence (1) and (2) 
hold for all j. 

Now part (2), combined with the HQ-avoiding condition, implies that 



1 00 


1* -j\ 


= 1 








1* -j\ 


> 2 



In other words, if |S(t>)| > 4 then Wsm is defined entirely by commuting rela- 
tions. We show that this hypothesis implies that |S(t>)| < 2. Indeed, suppose 
|S(f)| > 3, and let u = mim , where u\ G S ^W and Mo G Ws( v )- Here the set 
S ( V )W denotes the minimal length representatives of the left cosets W/Ws{ v )- By 
Equation ([7]), the product uqS2 ■ ■ ■ Si~v' must not be reduced. We conclude that 
Dr{uq) fl D L (s 2 ■ ■ ■ s^') = {s 2 } since for any Sj, Sj G S(u ) U S(f ) = S(t> ) we have 
m SiS . = 2 or 00. Moreover, since D L (sis 3 ■ ■ ■ s^') = {si,s 3 }, the same argument 
shows that s 3 G D R (u ). But now we have {s 2 , s 3 } C D R (u ) which implies that the 
m S2)SZ is finite. This contradicts the fact that m S2tS3 = 00. Hence, | S(v)\ < 3. 

Finally, if | S(v)\ = 3, then by Lemma [6. H m SlS2 ,m S2S3 > 3. If m slS3 = 2, then the 
HQ-avoiding condition implies m SlS2 = m S2S3 = 00. We can now apply the previous 
argument as above show that {s 2 , s 3 } C D r (uq) and hence m S2tS3 is finite. Thus we 
must have m SlS3 > 3. This completes the proof. □ 

For the next step in the proof of Theorem 13. 1[ suppose that |S(t>)| = 3 with 
S(v) = {r, s,t}. By Proposition I6.2[ we have m rs ,m rt ,m st > 3. Consider the reduced 
factorization v = xy where 

x^ 1 := tsrtsr ■ ■ ■ 

is the largest spiral word prefix of v. In other words, we can write 
(9) v = xy = (■ ■ ■ rstrst) ■ y. 

Define x' := xtst. It is easy to see that £(x') = £(x) — 1 and that x' equals x with the 
second to last reflection s removed. For any < k < £(x') define a length k suffix x' k 
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of x' by 

x' k := ; ■ ■ rstrt 

k 

Lemma 6.3. For any < k < £(x'), the following are true: 

(1) The product x' k y is a reduced factorization. 

(2) If k is even, then \D L (x' k y)\ = 1. If k is odd, then \D L (x' k y)\ < 2. 

(3) // \D L (x' k y)\ = 2 and k > 5, then \D L (x' k _ 2 y)\ = 2. 

Proof. If k — 0, then r, t ^ Dl{d) since a; is a maximal length spiral word. This 
implies that Di(y) = {s}. If k — 1, then t?/ is reduced and Diity) C {s, t}. Moreover, 
rt?/ is reduced and by Lemma |5\3| we have that D^irty) = {r} since r ^ Di(y). This 
proves the lemma for k < 2. 

We proceed with the proof by induction on k. Suppose k > 3. Without loss of 
generality, we can assume r 6 .D L (o4), so that s is the first element of x' k _ x . We first 
consider the case where k is odd. Then by the inductive assumption, we have that 
D L {x' k _ x y) = {s}. Hence 

x'kV = TX k-\V 

is reduced and D^{x' k y) C {r, s}. If is even, then s and t are the first two elements 
of in particular, r is not one of the first two elements. Therefore 

D L(x' k - 2 y) = {t} and D L{x' k _ x y) C {s,t}. 

So is reduced and D L (x' k y) = {r}. This proves parts (1) and (2) of the lemma. 
To prove part (3), suppose that k > 5 is odd with 

t E D L (x' k _ 2 y) C {t,r} and r G D L (x' k y) C {r, s}. 

If ^^(a;^?/)! = 2, then r G D L (x' k _ 2 y) since 3 < m rs < oo. Hence \D L (x' k _ 2 y)\ = 
2. " " ' □ 

One immediate consequence of Lemma 16.31 is that x'y is a reduced factorization 
and that if £(x') is even, then x'y G W J fl D(v ) which is a contradiction to Equation 
(|8]). Hence is odd (i.e. £(x) is even). The following lemma is a preliminary 
characterization of v. 

Lemma 6.4. The spiral word x satisfies one of the following conditions: 

(1) m rt = 3 and £(x) = 4. 

(2) m rt = m rs = 3 and £(x) = 6. 

(3) m rt = m rs = m st = 3 and £(x) > 8. 

Proof. Since x'y is reduced, we have that x'y ^ W J and £(x') > 3. Furthermore, 
by Lemma 16.31 part (3), \Di,(x' k y)\ = 2 for all k > 3. In particular the following 
statements are true: 

• For k = 3, we have that \DL(trty) \ = 2 if and only if m r t = 3. 

• For k = 5, we have that \DL(rstrty) \ = 2 if and only if m r t = m rs = 3. 

• For k = 7, we have that \DL(strstrty)\ = 2 if and only if m r t = m rs = m s t = 
3. 
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This completes the proof. □ 
Now we consider the reduced factorization 

x y 

(10) v = xy = (• • • rstrst) ■ y = (• • • rstr) (st st • • •) • y, 

length k 

where k is the length of the longest possible prefix of sty the form stst 

Lemma 6.5. With v as Equation (flQl) . the following are true: 

(1) y = e. 

(2) k = m st — 1 

Proof. Suppose that y ^ e. Then Di{y) = {r} by the maximality of k. If k = 2, 
then x is not a maximal length spiral, and hence y = e. Now assume that k > 3 
and let v = xzy be the reduced factorization given in (Till where z G W{ 8j t} is of 
length fc. Without loss of generality, let t G D R {z) and define z' := Since > 3, 
we have £(z') > 2 and thus xz'y is a reduced factorization. Likewise, since £(z') > 2 
and Dr(x) = {r}, we have that xz'y G W 7,7 and hence xz'y G W 7,7 fl D(v ). But this 
contradicts Equation ()8]). Therefore y = e and part (1) of the lemma is proved. 

Since £(x) is even, we have that k < m st , otherwise v ^ W J . This completes the 
proof in the case of x as in Lemma [6.41 part (3). Now suppose that k < m s t — 2. If x 
satisfies the condition in Lemma [6.41 part (1), then J = S(w) \ {t} and we can write 

v = tr gtst- • = trz. 

length k 

But then tz G W J fl D(t> ) which contradicts Equation ([8]). If x satisfies the condition 
in Lemma [6.41 part (2), then J = S(w) \ {r} and 

v = rstr stst^ - ■ - y = rstrz. 

length k 

But then rstz G W J fl D(v) which also contradicts Equation ([8]). Hence k > m st — 2 
and part (2) of the lemma is proved. □ 

It is easy to see that Lemmas 16. 3\ 16.41 and 16.51 prove the characterization v when 
| S(f)| = 3 in Theorem 13. II 

The final step in the proof is to show that w = uv is a BP-decomposition. In this 
step, we do not assume that \S{v) \ = 3. 

Lemma 6.6. For any So G S(v ) fl J, there exists v" G W J of length £(v") = £(v) — 2 
such that Sqv" G D(t> ). 

Proof. If | S(i>)| < 2, then the lemma is obvious. If | S(v)\ = 3, then we can write 
v = xy as in Equation (|9]) with the notational change that 

x = rstrst ■ ■ ■ . 

In other words, we let r, s, t denote the first three simple reflections appearing in x, 
rather than the last three. We want to find v" for s Q G S(t>) fl J = {s,t}. Note that 
with the change in notation, we have that m rs = 3. Recall the definition of x' given 
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after Equation ([9]). By Lemma [6731 part (1) we have that x'y is reduced and hence 
x'y G D(v ) fl W^ r ' s ^. Thus we have a reduced factorization 

x'y = (srs)y' 

for some y' . For s = s, we set v" = rsy' . Then v" G W^ 7 since Di(v") = {r}. 
We now find a u" for so = t. Consider the reduced factorization 

v = (rs)(ty"). 

Clearly r ^ D L (ty"), otherwise v W J . Hence rty" G D(v) and rty" ^ iy J . This 
implies that t G Di{rty") and we can write a reduced factorization 

rty" = (trt)y"' 

for some j/'". We set v" = rty'" for s = £• Since Dl(v") = {r}, we get that v" G IV J . 
This completes the proof. □ 

If s G S(v) fl J and u" G W 7,7 , such that s v" G D(u), then s G D R (u). Otherwise 
usqv" G m • D(t>) fl D(w) which contradicts Equation (j7|). Applying Lemma [2.2[ we 
get that w = uv is a BP-decomposition. This completes the proof of Theorem 13.11 
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